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Abstract. When trying to discover, assess, and select cloud services, 
companies face many challenges, snch as fast-moving markets, vast num¬ 
bers of offerings, and highly ambiguons selection criteria. This publica¬ 
tion presents the Open Service Compendinm (OSC), an information sys¬ 
tem which snpports businesses in their discovery, assessment and clond 
service selection by offering a simple dynamic service description lan¬ 
guage, business-pertinent vocabularies, as well as matchmaking fnnction- 
ality. It contributes to the state of the art by offering a more practical, 
mature, simple, and nsable approach than related works. 
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1 Introduction 

There is a major trend within enterprise IT to fundamentally embrace cloud 
computing. The most recent 2015 ’’State of the Cloud Survey” reveals that 93 
percent of large enterprises (i.e. 1000-1- employees) are already using cloud com¬ 
puting solutions, 82 percent follow a multi-cloud strategy, while only 3 percent 
do not have plans for adopting cloud computing 

Before companies contract and consume cloud services, they have to carry 
out discovery, i.e., finding cloud services in the vast Internet, assessment, i.e., 
matching services to requirements, and selection, i.e, choosing the best service for 
subsequent booking and consumption, e.g., by making a shortlist and ranking 
services. These tasks are challenging: cloud markets are fast-moving, have a 
vast numbers of offerings, selection criteria are highly ambiguous, marketplaces 
sometimes unorganized, and price structures and feature combinations complex 
and opaque. These challenges impede optimal service selection and sometimes 
hinders cloud adoption generally. 

Our contribution was conceived within two research projects targeting spe¬ 
cific domains: TRESOF0 targeting the German Health sector and CYCLONEj^ 

^ http://goo.gl/eloh66 
^ http://www.cloud-tresor.com 
^ http://www.cyclone-project.eu 



targeting users of federated, multi-cloud applications. TRESOR developed a 
cloud ecosystem featuring a cloud broker & marketplace and was thoroughly 
presented in our previous works \2im\2tim\ . CYCLONE is a Horizon 2020 in¬ 
novation action which aims at integrating existing cloud management software to 
allow unified management of federated clouds. Both projects also address discov¬ 
ery, assessment, and selection challenges due to the lack of a suitable information 
systems: The TRESOR health centers cannot assess legal cloud consumption pre¬ 
requisites, e.g., how and where medical data is processed. This leads to higher 
costs for local IT infrastructure and less functionality available to personnel. 
Many of the CYCLONE multi-cloud application developers face challenges in 
selecting optimal offerings for use in their applications, e.g., laaS VMs and stor¬ 
age services. Suboptimal offerings can cause higher costs as well as lower Quality 
of Experience by the end-users of such applications. 

Our previous work [5D] establishes basic technologies for addressing these is¬ 
sues: a textual domain specific language for describing services, a pertinent busi¬ 
ness vocabulary of selection criteria, and a brokering component. The analysis of 
related work showed a particular lack of pertinent service selection criteria in de¬ 
scription languages as well as contemporary and future marketplaces, although 
there has been extensive empiric research in this area. Also, the benefits of using 
textual domain-specihc languages [5] is not utilized in any of the examined ap¬ 
proaches, which predominantly use semantic technologies for capturing service 
information. The main contribution of this publication is the design, implemen¬ 
tation, and evaluation of a first iteration of the OSC, which is an information 
system supporting business users in their cloud service discovery, assessment, 
and selection activities. For this, we employ and extend former contributions 
and address the following research questions in this publication: 

Q1 What are the main business challenges the OSC has to address and where 

should it differentiate itself from the state-of-the-art and related works? 

Q2 Which use cases should the OSC architecture implement and how should it 

be designed to be suitable, scalable, and state-of-the-art? 

Q3 How does the current OSC implementation meet its requirements, as well as 

the needs of its first users? 

By addressing these research questions we extend existing research on de¬ 
scription languages, matchmakers, and marketplaces, by including real-world 
requirements to further maturate this area of research. By designing the system 
to be ” wiki-like” and having it used by regular Internet users we hope to increase 
the volume of empiric knowledge. 

We apply the Information Systems Research Framework by Hevner et al. 
(Ill) which also structures this publication: Chapter iterates prevalent busi¬ 
ness challenges and derives eight main OSC requirements. These requirements 
are contrasted with the related work in Chapter to guide the OSC use case 
definition in Chapter]^ Based on these use cases we design the OSC architecture 
in Chapter and present its current implementation status. After showing first 
evaluation results in Chapter we conclude this publication in Chapter 




2 Cloud Service business challenges 


This chapter structures the discovery, assessment, and selection challenges into 
three problem areas, describes them, and specihes requirements for the OSC, 
numbered R1 to R8. 

Cloud Market Characteristics: Fast-moving Vastness. The cloud mar¬ 
ket is vast and fast-moving: Current forecasts demonstrate its increasing vast¬ 
ness: the total end-user spending on public cloud services is expected to grow 
by almost 60% between 2015 and 2018 to a staggering $290bi|^ Some cloud 
vendors are also astonishingly large: Amazon Web Services, for example, has 
more than 1 million customers, achieved more than 40 percent year-over-year 
revenue growth, and generates an estimated yearly revenue of $4 billion The 
’’Google Memorial”!^ highlights the velocity of a fast-moving cloud market par¬ 
ticipant: it lists 66 discontinued services which were sometimes highly popular, 
for example, the Google Reader service had more than 24 million user^ before 
it was suddenly discontinued in 2013. These examples highlight that the cloud 
market is too vast for companies to obtain an optimal overview and it is too 
fast-moving to keep up with ever-changing service offerings. These cloud market 
characteristics require a structured service repository (Rl), which integrates dy¬ 
namic information (R2), e.g., laaS spot-market prices. For maximum impact, 
it should be ”wiki-like”, i.e., any Internet user should be able to create and edit 
service descriptions (R3). A matchmaking between requirements and contained 
knowledge should support service selection (R4). 

Ambiguous criteria and scattered knowledge. Assessing service offer¬ 
ings raises two questions: what criteria to use and where to get the required 
information. Deciding what criteria to use is hard: they are sometimes highly 
ambiguous (e.g., data privacy criteria as shown by Selzer m) and sometimes 
empirically identihed, yet neither integrated into service description languages, 
nor existing marketplaces and repositories, as we’ll show in the next chapter. 
Gathering information to apply these criteria is also a challenging task: First of 
all, companies conceal knowledge about unfavorable service aspects. For exam¬ 
ple, cloud backup providers label services ’’unlimited”, while they have in fact 
bandwidth and storage limits]^ The ’’Fair Use” clause of Backblaze, which allows 
the provider to cancel the contract anyti me Pj and the GrashPlan ’’Unlimited” 
limits which are concealed in the EULA highlight this practice. Secondly, 
some companies provide insufficient information: for example, Microsoft states 
that OneDrive can only be used with the Windows 8.1 Explorer if users use 
their live.com accounts for logging on to Windows On the contrary, many 

http: //www. ft. com/cms/s/2/b3d40e7a-ceea-lle3-ac8d-00144f eabdcO .html 
® http://goo.gl/5vHSom 
® http://goo.gl/YdN2np 

^ http://googlesystem.blogspot.de/2013/03/google-reader-data-points.html 
® http://goo.gl/nVqeA3 
® https://www.backblaze.com/terms.html 
http://support.code42.com/CrashPlan/CrashPlan_For_Home_EULA 
http://windows.microsoft.com/en-us/windows-8/onedrive-app-faq 



companies are reluctant to allow their corporate users to use such accounts, thus 
hindering them to use OneDrive effectively. Only private blogs offer workarounds 
which are not always discovered by companies wishing to assess OneDrive 13 
In summary, as provider information does not suffice and knowledge becomes 
more scattered, the efforts for assessing services rise constantly unless there is a 
vocabulary of selection criteria pertinent to businesses (R5), as well as means 
of integrating external information (R6). 

Features and Prices: Complex and Incomparable In his seminal 1956 
paper, Smith outlined that product differentiation and market segmentation are 
viable marketing strategies [22] • This observation still holds true almost sixty 
years later: to compete with cloud market leaders, service providers differentiate 
products and segment their market. One example is the online storage market, 
which is segmented into related categories, such as ’’remote backup”, ’’cloud 
storage”, and ’’file sharing”. Different needs of consumers are addressed by dif¬ 
ferent features and pricing schemes. ’’Cloud storage” services, such as Google 
Drive, allow flexible sharing of data, but incur additional costs for extending 
the free quota. ’’Backup services”, such as CrashPlan, allow ’’unlimited” data 
storage for a fixed price but have only limited sharing functionality, e.g., backup 
family plans, such as ’’CrashPlan for Home”. Thus, comparing different services 
becomes challenging if cloud consumers need to both share and backup large 
volumes of data. The price structure and feature combinations can also become 
complex: for example, Amazon EC2 offers 32 VM types in 10 locations with 6 
operating systems, resulting in 1920 configuration options to choose from; in ad¬ 
dition to opting for either on-demand, reserved, or spot market instances. Thus, 
comparing different competitors to find an optimal product implies an enormous 
effort, unless there is a suitable price model (R7) as well as a mechanism for 
describing different variants of a service (R8). 


3 Related Work 

The preceding business challenges are addressed by a number of related works 
from academia and practitioners in the areas of service deseription languages, 
repositories and marketplaces, serviee matchmaking approaches, as well as cloud- 
selector frameworks. 

Service Description Languages. Dervice description languages capture 
elevant aspects of services for a specific use case. For example, WSDlj^ and 
the CORBA IDL describe the technical service interface for the main use case 
of automated code generation of service skeletons. There is a wealth of service 
description approaches in the field of semantic web services, e.g., OWL-S [12] . 
WSMO [m, SAWSDlj^ WSDL-S[5 SWSlf^ and others. Other languages 

http://goo.gl/PZ7d6p 

http://www.w3.org/TR/wsdl20/ 

http: //WWW . w3 . org/TR/2007/REC-sawsdl-20070828/ 
http://www.w3.org/Submission/WSDL-S 
http://www.w3.org/Submission/SWSF/ 



focus on business-related information, such as WSM04IoS [23] as well as the 
Linked-USDL [15] which is derived from the earlier USDL M- Other researchers, 
e.g., Breskovic, et ah, create standardized products for electronic markets [3], 
based on description languages, such as the CRDL m- At last, some authors 
focus on price and cost modeling of cloud services (R7), for example Kashef and 
Altmann in [9] and [1]. While these works provide interesting application areas 
for the OSC, they are not based on an existing service description language. 

Seminal works by Fensel, et al. [S] as well as Studer, et al. [55] present seman¬ 
tic web services in detail. Studer, et al. summarizes the focus areas of seman¬ 
tic web services: reasoning-based matching of service functionality, harmonizing 
data formats and protocols, and automated Web Service composition. Seman¬ 
tic functionality requires service knowledge expression, e.g., service inputs and 
outputs, preconditions, and service effects. Therefore related languages have a 
broad scope and aims: for example, ’’maximise to the extent possible the level 
of automation” [T5] or ’’covering as many XaaS domains as possible” [53]. In 
contrast, cloud service discovery, assessment and selection activities by SMEs 
require merely a small set of relevant information, which has to be pertinent to 
business users. Yet, no approach meets the business challenges sufficiently: they 
do not handle dynamic information (R2) well, cannot be used wiki-like (R3), 
nor can easily integrate external information (R6). While there are empiric 
studies on service selection criteria, e.g., [T5] and the CloudServiceChecIp^ the 
languages do not capture such service knowledge pertinent to businesses (R5). 
Another issue is the missing service variant managemenif^ Without having a 
rich variant model, describing real-world cloud services becomes a major chal¬ 
lenge, demonstrated by the example Amazon EC2 Linked-USDL descriptior]^ 
It only considers one type of instance and only one location, but consists of 1899 
lines. We approximate a complete EC2 description to be 300.000 lines in length. 
This shows the prohibitive complexity of real-world USDL descriptions leading 
to inefficient processing and low human comprehensibility. 

We argue that the failure to address existing SME challenges is the reason for 
its missing industry adoption outside of their funding scope. Many languages are 
abandoned, e.g., OWL-S (2006), SAWSDL (2007), WSMO (2008), and USDL 
(2011), and the associated tools are not updated anymore, e.g., the WSMO Stu- 
dicj^ Therefore we do not consider Semantic Web Service approaches suitable 
as the basis of the Open Service Compendium. 

Repositories and Marketplaces address the vastness of the Cloud mar¬ 
ket by managing a large number of service descriptions. They can be divided 
into academic marketplace research platforms and high-volume SaaS market¬ 
places. There are academic marketplace research platforms which are relevant 
to our contribution: The USDL marketplacj^ a proof-of-concept marketplace 
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The USDL ’’variant management” connotes variants of the language. 
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prototype based on USDL. The FI-Ware Marketplace and Repositorjj^ provide 
APIs to manage USDL service descriptions, as well as support discovering and 
matching application and service offerings. At last, Spillner and Schill offer an 
extensible XaaS Service Registry which is based on WSMOdloS [21] . 

We observe that no approach overcomes the explicated limitations of its SDL. 
Furthermore, none is designed to be used by regular Internet users, thus hin¬ 
dering their broad adoption and lowering their pertinence to businesses. Proto¬ 
typical high-volume commercial SaaS marketplaces are the Google Marketplace 
and Salesforce AppExchange Instead of an elaborate cloud service for¬ 
malization, they utilize very simple data models consisting of free-text, images, 
provider info, and a categorization. As they lack a service formalization, they 
are only marginally able to support users in their service discovery, selection, 
and assessment. 

Service Matchmaking. The related work on service matchmaking is highly 
divergent in its contexts (e.g., Cloud Services, SOA, the Semantic Web), as well 
as on its opinions about what constitutes a matchmaking problem (e.g., the types 
of variables and the desired matchmaker functionality). We examine, if the ex¬ 
isting service matchmakers are business-pertinent (R5). Our previous survey 
|29j divides approaches into syntactic, constraint based, ontological and Fuzzy 
Set Theory based: Syntactic approaches are limited to numeric QoS parameters 
BUD]. Constraint based transforming the service request into a set of constraints 
and match it to a set of service descriptions [10]. Afterwards, the ’’closest” ser¬ 
vice can be found using the Euclidean distance between the request and the 
description [3]. Ontological approaches utilize OWL-S and reasoners, for exam¬ 
ple, to calculate the semantic similarity of method signatures m, and to define 
the constraints as SWRL rules [8]. Fuzzy Set Theory based approaches aim to 
match numeric QoS parameters in a flexible manner, sometimes extending syn¬ 
tactic approaches, for example, by allowing ’’good”, ’’medium”, and ’’poor” value 
intervals [ig, or using trapezoidal fuzzy numbers [2]. 

While being extensive, none of the related approaches addresses current busi¬ 
ness challenges: most of them only consider numeric QoS parameters, such as 
availability, response time, and throughput which are not independently and 
objectively measurable. Eurthermore, pertinent selection criteria which are not 
numeric, e.g., feature lists, are not considered, as was shown in our previous 
publication [30]. The referenced empiric studies show that only a small subset 
of relevant selection criteria are numeric. At last, none of these approaches uses 
more sophisticated description formats than WSDL. 

Cloud-selector frameworks help cloud users in assessing different aspects 
of cloud providers. One example is PlanForClouij^ which allows users to create 
deployment descriptions and specify their planned usage of cloud resources, e.g., 

https://github.com/service-business-framework 
https://www.google.com/enterprise/marketplace/home/apps 
^ https://appexchange.salesforce.com 
http: //www.uoguelph. ca/~qmahmoud/qws/ 
https://planforcloud.rightscale.com 





servers, storage, and databases. CloudHarmonjj^is a bundle of services by Gart¬ 
ner: a provider directory, a benchmark database for network performance, and 
a service status dashboard. CloudSpectatoi[^ offers performance measurements 
for different laaS providers. Many limitations persist: PlanForCloud contains 
only services supported by RightScale software. CloudHarmony is quite exten¬ 
sive, yet lacks information about pricing and other business-pertinent selection 
criteria. None of the platforms offers matchmaking functionality or crowdsourc¬ 
ing data. The criteria are also limited, e.g., PlanForCloud offers only ’’price”, 
while CloudSpectator supports only a ’’Multi Core Score”. 

4 OSC Use-cases 

This chapter derives the OSC use-cases which address the business challenges 
and their requirements illustrated in the preceding chapter. The use-cases are 
numbered U1 to U6 and are described in detail while Figure provides the UML 
use-case diagram for easy reference. 



Fig. 1. OSC Use Cases 

Ul: Manage Service Descriptions. The OSC should provide functionality 
for Internet users to manage service descriptions, i.e, to create, show, edit, and 
delete structured service descriptions to provide a structured service repository 
(Rl). As the OSC is ”wiki-like” (R3), the use-case should allow a comprehensible, 
text-based language for service descriptions. 

U2: Create Vocabulary. The business-pertinent vocabulary (R5) provides 
the structure for service descriptions managed by Use Case Ul. Thus, the OSC 
should provide Internet users the functionality to create such a vocabulary. To 
preserve the ”wiki-like” characteristic of the OSC, the vocabulary definition 
has to be based on a comprehensible, text-based language. In order to support 
companies with their assessment, the vocabulary should also contain additional 
information on why a certain property is important for service selection. 

U3: Program dynamic service information updates. To integrate ex¬ 
ternal information (R6) and to also allow service knowledge to be dynamically 
updated (R2), the OSC should give Internet users with programming knowledge 
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the ability to implement dynamic service information updates. For reduced com¬ 
plexity, dynamic and static parts of a service descriptions should be integrated 
tightly, e.g., by having both in the same document. Dynamically updated service 
descriptions can lower the authoring effort considerably. 

U4 & U5: Manage price models and service variants. In order to han¬ 
dle complex price and variant structures, the OSC should manage price models 
(R7) as well as variants (R8) of services including creating, showing, editing 
and deleting these models, as well as presenting a price calculator, a variants 
overview, and using the price and feature model within service comparisons. 
The OSC should evaluate these models when finding services, i.e., it should find 
the respective variant of a service matching the search request as well as cal¬ 
culating the prices of those variants. This model should be integrated into the 
description language in order to reduce complexity and to allow features to be 
linked to their impact on the price of service consumption. 

U6: Find matching services. All of the previous use-cases culminate in 
the pivotal use-case of enabling Internet users to find matching services to their 
requirements (R5). For this, end-users should be able to define their requirements 
in an interactive fashion. The OSC should then evaluate services, price, and 
feature models to present matching services. In addition to basic selection, the 
OSC should also contain a matchmaking component in order to rank services 
and provide a more comprehensive selection result. 

5 Open Service Compendium Architecture 

This chapter presents the Open Service Compendium architecture which is de¬ 
signed to implement the OSC use cases by providing insight on three layers: the 
conceptual architecture, its implementation, as well as its future expansion. 

The conceptual architecture. Figure [^presents the conceptual architec¬ 
ture in form of a UML component diagram. There are six main components, 
which are described in the following paragraphs. 

The pivotal OSC controller is responsible for providing a service list as 
JSON to the Frontend^ either by querying the Database or using the Cache. 
Changes to service descriptions are recieved from the Frontend in SDL-NG form 
and submitted to the Job Queue for subsequent evaluation. The Job Queue can 
additionally save the evaluation status, e.g., in cases of erroneous SDL-NG docu¬ 
ments. The Service Evaluator takes SDL-NG documents from the Job Queue, 
executes them in a specially secured container to prevent malicious code execu¬ 
tion, and writes the resulting service descriptions to the database. The Frontend 
provides the end-user interface for the use cases U1 through U6. It is designed as 
a JavaScript ’’Single Page Application” and is responsible for querying the OSG 
controller for JSON service descriptions, service matchmaking, service variant 
handling, price calculation, as well as submitting new and modified service de¬ 
scriptions to the OSC controller. A number of factors led to our adoption of the 
emerging ’’Single Page Application” style, instead of having the OSC controller 
render all the pages: the back end simplicity, the swiftness of the user interface, 



Fig. 2. Conceptual architecture 

as well as the decoupling of back end and front end. As our architecture is well 
decoupled, each component can be scaled independently. 

Implementation. The OSC prototype can be accessed online and its 
current source code can be found on GithufP^ While we describe the final im¬ 
plementation, some of the components are currently under development and not 
publicly available for testing. 

We used the brokering component of the TRESOR project as the base for 
creating the OSC controller, a Ruby on Rails application offering RESTful APIs 
for creating, querying, updating, and deleting services stored in a MongoDB 
database. The database holds service documents which contain a persisted rep¬ 
resentation of the executed ruby service description as well as meta-information, 
such as the execution timestamp. The Job Queue uses a Redis key-value store 
managed by the Resque Ruby library. For caching, we rely on the Rails-default 
ActiveSupport: : Cache infrastructure, as it is highly flexible in its use of dif¬ 
ferent caching stores, for example, in-memory, plain files, MemCache, or Redis. 
The service evaluator is a regular Ruby process using the SDL-NG to evaluate 
the service descriptions in a secure context and persisting the resulting service 
descriptions in the database. The SDL-NG contains a language infrastructure 
(e.g., a type system and exporters), utility classes for scraping websites, business 
vocabularies (e.g, for cloud storage and laaS offerings), as well as a price and 
feature model. As SDL-NG descriptions can scrape websites with changing con¬ 
tent, e.g., Amazon Spot Market prices, the service evaluator can be instructed 
to regularity check the description for changes and update the database records 
accordingly. This functionality can handle dynamic aspects of cloud systems, 
e.g., pricing and capability changes. Historical service records can be retrieved 
using the OSC controller, for example, to detect modifications of certain services 
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and to make predictions about future changes (e.g., price drops). Our previous 
publication [5D] provides an in-depth explanation of the SDL-NG. 


Search « Service comparison 



Fig. 3. User interface: faceted search, service comparison, and storage vocabulary 

The frontend is built and assembled using a Grunt JavaScript workflow. It 
is based on AngularJS and a number of additional Javascript and CSS libraries, 
e.g., Angular UI Router, Twitter Bootstrap, Less, SASS, and CoffeeScript. It in¬ 
tegrates a Java applet containing a constraint programming matchmaker using 
the Choco Solveij^to implement different constraint models for different match¬ 
making functionality: discrete value matching with hard and soft constraints, 
interval matching for negative and positive tendencies, and matching of feature 
lists. Our previous work [SD] includes a detailed description of the constraint 
models and their implementation. Figure]^ shows some example screenshots of 
the user interface. In general, there are three main views for users to realize 
service discovery, assessment, and selection. First, a list of all services in a spe¬ 
cific category (e.g., cloud storage and laaS solutions) allows users to discover all 
available services. Users can use a faceted search to filter the list based on their 
selection criteria, such as company jurisdiction, payment options, and certifica¬ 
tions. For assessment there is a ’’detail” view, where users can see the whole 
service description, including extensive documentation about all properties and 
their meaning, as well as a comparison view. To support their final selection, 
users can employ the matchmaking component for ranking services based on 
user defined constraints and getting the ’’best” service for their needs. 

Future expansion. Through eventual OSG advancements, we foresee some 
prospective components: first of all, there has to be some kind of basic user 
management to protect descriptions from vandalism or malicious editing. Sec¬ 
ondly, to strengthen the usefulness of the OSC, additional external information 
sources should be included and managed by OSC components, e.g., external 
service reviews, user ratings, and benchmarking data. At last, the RDF/OWL 
export capabilities of the Service Description Language could be used to im¬ 
plement a Semantic Data Store component in order to publish an OSC dataset 

http://www.emn.fr/z-info/choco-solver/ 











in the Linked Open Data Cloud. This has two main goals: raising the business 
relevance of related approaches by offering semantic descriptions of real-world 
services as well as enabling the OSC to benefit from advanced functionality, such 
as machine learning and reasoning. 


6 Evaluation 

This chapter presents the evaluation of the OSC architecture and its implemen¬ 
tation: analytical, comparing it to the set of general requirements, experimental, 
gathering knowledge from using it in practice, as well as empirical, carrying out 
interviews and surveys. 

Analytical Evaluation. Carrying out the analytical evaluation is straight¬ 
forward and highlights the fitness of the OSC to cover all enumerated require¬ 
ments: we have created a structured service repository (Rl) using a comprehen¬ 
sive Service Description Language. Dynamic information (R2) can be contin¬ 
uously integrated by frequently running the Service Evaluator, as explained in 
Chapter As we have chosen a simple to use textual DSL, the OSC becomes a 
”wiki-like” information system (R3), which was also highlighted in our recent 
publication |20| . We integrated a service matchmaker (R4) to match require¬ 
ments with structured service knowledge. The business and category-specific vo¬ 
cabularies contain empirically determined selection criteria, which are pertinent 
to businesses (R5), which is highlighted by our empirical evaluation. Internet 
Users with programming knowledge can integrate external information (R6) 
using the Utility Classes of the SDL, as exemplified in the service description 
examplej^ The SDL-NG contains an additional price model (R7) as well as a 
model to capture service variants (R8). 

For experimental evaluation, we have applied the OSC in practice to 
validate its functionality by implementing an automated test suite, as well as 
manual usage. So far, the OSC functions as specified. For example, users can 
have a look at the vocabulary ’’cheat sheet” to get an overview of all properties 
and types, use a code editor to create service descriptions, view the output of 
the service evaluation, search for services, and compare them. 

Additional empirical evaluation was carried out for the OSC components 
service description language, business voeabulary, and the cloud storage vocab¬ 
ulary. The service description language was presented to a group of experts 
from other Trusted Cloud projects having been involved in related research, 
e.g., the USDL [T3]. A group discussion about the relevance of the OSC for the 
field resulted in the following statements: the textual DSL is a major simplifica¬ 
tion in describing services, especially in comparison to the USDL and semantic 
approaches. Deriving the vocabulary from empiric research strengthens the use¬ 
fulness of the DSL in practical contexts. The utility value of a central repository 
with matchmaking capabilities was regarded as very high. An expert group eval¬ 
uated the business vocabulary with respect to its relevance for service selection. 
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Participants were one publication author, the CIO, and two IT project managers 
of a large German health center. They had to come up with a mutual impor¬ 
tance rating of the individual criteria on a 5-step scale from ’’indispensible” (1) 
to ’’irrelevant” (5). The left pie chart in Figure [^groups the categories by their 
rated importance: 86.5% of the 52 criteria were rated important and higher, 
while only 13.5% were rated less important or irrelevant. Evaluation of an inter¬ 
mediate version of the Cloud Storage Vocabulary was performed using an online 
questionnaire in which participants rated the importance of the 27 criteria for 
their selection of a cloud storage service. The respondents were mostly students 
of computer science and related fields of study, providing valuable insight into 
the usefulness for generic Internet users, as most other people involved in our 
research are either professionals or academics. Of 35 respondents, 18 (51.4%) 
completed the questionnaire. The right pie chart in Figure shows the distri¬ 
bution of the average importance of all criteria, which is grouped into 1 — 1.5 
(indispensable), 1.5 — 2.5 (very important), and 2.5 — 3.5 (important). These 
results show that the generic OSC business vocabulary is able to capture some 
of the most important selection criteria of this specific client. For generalization, 
we will conduct an extensive questionnaire in the future and adjust the business 
vocabulary according to its results. The results promise a high relevance of the 
vocabulary for Internet users, yet the absence of less important and irrelevant 
criteria could also point at the inability of our respondents to differentiate the 
importance of their criteria. The low completion rate could imply that people 
either could not understand the criteria or did not know their cloud provider 
selection process. 



□ Indispensible 

□ Very Important 

□ Important 


Fig. 4. Survey results for business (left) and cloud storage vocabulary (right) 


7 Conclusion 

We have delineated the challenges in discovery, assessment, and selection of cloud 
services and revealed the failure of both academic and commercial approaches to 
address these challenges properly. By using real-world requirements as the basis 
for the OSC use cases as well as designing a modern solution architecture, we 
hope to create a practical, mature, simple and usable information system. Prelim¬ 
inary evaluation results are promising and we are looking forward to presenting 
and discussing our contribution with practitioners and researchers at GECON. 
We also hope to maximize the impact of the OSC by creating a ” wiki-like” infor¬ 
mation system and publishing it as free and open source (EOSS) software. In the 


future, we will use the OSC as a basis to tackle upcoming challenges within fed¬ 
erated, multi-cloud environments and the Intercloud in the CYCLONE project. 
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